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METHOD FOR TASK CLASSIFICATION USING 
MORPHEMES 

5 

This non-provisional application claims the benefit of U.S. Provisional 
Patent Application No. 60/163.838 . filed November 5, 1999, which is 
incorporated herein by reference in its entirety. 
Technical Field 

1 0 The invention relates to automated systems for communication 

recognition and understanding. 

Background Of The Invention 

Conventional methods for constructing spoken language systems involve 
collecting and annotating large speech corpora for a task. This speech is 

1 5 manually transcribed and each utterance is then semantically labeled. The 
resultant database is exploited to train stochastic language models for 
recognition and understanding. These models are further adapted for different 
dialog states. Examples of such methods are shown in U.S. Patent Nos. 
5,675,707, 5,860,063 and 6,044,337, and U.S. Patent Application Nos. 

20 08/943,944, filed October 3, 1 997, and 09/21 7,635, filed December 21 , 1 998, 
each of which is incorporated by reference herein in its entirety. 

This transcription and labeling process is a major bottleneck in new 
application development and refinement of existing ones. For incremental 
training of a deployed natural spoken dialog system, current technology would 
25 potentially require transcribing millions of transactions. This process is both time- 
consuming and prohibitively expensive. 

Summary Of The Invention 

The invention concerns a method of task classification using morphemes 
which operates on the task objective of a user. The morphemes may be 
30 generated by clustering selected ones of the salient sub-morphemes selected 
from training speech which are semantically and syntactically similar. The 
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method may include detecting morphemes present in the user's input 
communication, and making task-type classification decisions based on the 
detected morphemes in the user's input communication. The training speech 
may be verbal, non-verbal, a combination of verbal and non-verbal or multimodal. 
Brief Description of the Drawings 

The invention is described in detail with reference to the following 
drawings wherein like numerals reference like elements, and wherein: 

Fig. 1 is a block diagram of an exemplary speech recognition system 
using acoustic morphemes; 

Fig. 2 is a detailed block diagram of an exemplary acoustic morpheme 
generator; 

Fig. 3 is a flowchart illustrating an exemplary acoustic morpheme 
generation process; 

Fig. 4 is a flowchart illustrating an exemplary candidate phone-phrase 
selection process; 

Fig. 5 is a flowchart of an exemplary speech recognition and task 
classification process using acoustic morphemes; 

Fig. 6 is a graph illustrating the number of recognized phones per 
utterance; 

Fig. 7 is a graph illustrating the length comparison of recognized vs. 
transcribed utterances; 

Fig. 8 is a graph illustrating the mutual information (Ml) of phone-phrases, 
showing increased Ml as their length increases; 

Fig. 9 is a graph illustrating the P max of phone-phrases, showing more 
phrases with high P max as their length increases; 

Fig. 10 illustrates examples of salient phone-phrases for "collect"; 

Fig. 1 1 illustrates an example of an acoustic morpheme for "collect"; 
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Fig. 12 is a graph illustrating the length of salient phone-phrases; and 

Fig. 13 is a graph illustrating the experimental call-classification 
performance on speech using acoustic morphemes. 

Detailed Description Of The Preferred Embodiments 

5 This invention concerns a spoken dialog system that automatically learns 

from speech without transcriptions. Semantic labels can be extracted 
automatically from either experiments or from autonomous dialogs. In particular, 
a task-independent phone-recognizer is taught how to 'learn to understand' from 
a database of untranscribed (or transcribed) speech plus semantic labels. 

1 0 Baseline approaches to the teaching of a speech recognition systems to 

understand are found in U.S. Patents Nos. 5,675,707 and 5,860,063, and U.S. 
Patent Applications Nos. 08/943,944 and 09/217,635, which are incorporated 
herein by reference in their entireties. 

The earliest work demonstrated automatic acquisition of 'words' and 
1 5 'grammar' from collapsed text. That work did not address, however, the issues 
arising from non-perfect recognition of speech. The next step was to show how 
to acquire lexical units from speech alone without transcription and exploit them 
for spoken language understanding (SLU). That experiment, however, was 
constrained to speech comprising isolated word sequences and used matching 
20 techniques to decide if an observation was a new 'word' or variation of a known 
'word'. 

All of the above efforts involve learning from speech alone. While one can 
learn much about a spoken language by merely listening to it, the process can 
progress further and faster by exploiting semantics. This has been demonstrated 
25 in both the engineering domain and in analyses of children's language 

acquisition. Thus, this invention goes beyond the past efforts by exploiting 
speech plus meaning using morphemes, both acoustic and non-acoustic, in order 
to teach a machine to 'learn to understand'. 
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While the morphemes may be non-acoustic (i.e., made up of non-verbal 
sub-morphemes such as tablet strokes, gestures, body movements, etc.), for 
ease of discussion, the systems and methods illustrated in the drawings and 
discussed in the below concern only acoustic morphemes. Consequently, the 
5 invention should not be limited to just acoustic morphemes and should 
encompass the utilization of any sub-units of any known or future method of 
communication for the purposes of recognition and understanding. 

Furthermore, while the terms "speech", "phrase" and "utterance", used 
throughout the description below, may connote only spoken language, it is 
1 0 important to note in the context of this invention, "speech", "phrase" and 

"utterance" may include verbal and/or non-verbal sub-units (or sub-morphemes). 
Therefore, "speech", "phrase" and "utterance" may comprise non-verbal sub- 
units, verbal sub-units or a combination of verbal and non-verbal sub-units within 
the sprit and scope of this invention. 

15 Fig. 1 is an exemplary block diagram of a possible speech recognition and 

understanding system 100 that utilizes acoustic morphemes. The speech 
recognition and understanding system 100 includes two related subsystems, 
namely an acoustic morpheme generation subsystem 1 10 and input speech 
classification subsystem 120. 

20 The acoustic morpheme generation subsystem 1 1 0 includes an acoustic 

morpheme generator 130 and an acoustic morpheme database 140. The 
acoustic morpheme generator 130 generates acoustic morphemes from a corpus 
of untranscribed training speech (the invention may also operate with training 
speech that is transcribed, of course). The generated acoustic morphemes are 

25 stored in the acoustic morpheme database 140 for use by the acoustic 

morpheme detector 160. The acoustic morpheme database 140 contains a large 
number of verbal and non-verbal speech fragments or sub-morphemes 
(illustrated as phone-phrases for ease of discussion), each of which is related to 
one of a predetermined set of task objectives. Each of the acoustic morphemes 

30 may be labeled with its associated task objective. The operation of the acoustic 
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morpheme generator 130 will described in greater detail with respect to Figs. 2-4 
below. 

The input speech classification subsystem 120 includes an input speech 
recognizer 150, an acoustic morpheme detector 160 and a task classification 
5 processor 170. The input speech recognizer 1 50 receives a user's task objective 
request in the form of verbal and/or non-verbal speech. The input speech 
recognizer 150 may be of any known design and performs the function of 
recognizing, or spotting, the existence of one or more acoustic morpheme in the 
user's input speech. 

1 0 The acoustic morpheme detector 1 60 then detects the acoustic 

morphemes present in the user's input request. The acoustic morphemes 
generated by the acoustic morpheme generation subsystem 1 10 are provided as 
an input to acoustic morpheme detector 160. 

The output of acoustic morpheme detector 160 includes the detected 

1 5 acoustic morphemes appearing in the user's task objective request that is then 
provided to the task classification processor 170. The task classification 
processor 170 may apply a confidence function, based on the probabilistic 
relation between the recognized acoustic morphemes and selected task 
objectives, and makes a decision either to implement a particular task objective, 

20 or makes a determination that no decision is likely in which case the user may be 
defaulted to a human or automated system for assistance. 

An exemplary process of the invention will now be described with 
reference to Figs. 2-4. Fig. 2 is a detailed block diagram of an exemplary 
acoustic morpheme generator 130. The acoustic morpheme generator 130 

25 includes an ASR phone recognizer 21 0, a salient phone-phrase generator 220 
and a clustering device 230. 

Fig. 3 illustrates a possible process of generating acoustic morphemes 
using the acoustic morpheme of generator 130 of Fig. 2. The process begins at 
step 3000 and proceeds to step 3100 where the ASR phone recognizer 210 

30 receives raw training speech from a database, for example. The database may 
be generated from recordings of users talking with human agents, responding to 
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the prompt "AT&T. How may I help you? (HMIHY). The characteristics of this 
data and early experiments are detailed in U.S. Patent No. 5,675,707, for 
example. 

In an embodiment for recognizing non-acoustic morphemes, the ASR 
5 phone recognizer 210 would be replaced in the figure by a sub-morpheme 
recognizer. The sub-morpheme recognizer would operate similar to the ASR 
phone recognizer, but it would receive raw non-acoustic or a mixture of acoustic 
and non-acoustic training data from a database. Again, for each of discussion, 
only acoustic morphemes will be discussed. 
10 A training set of thousands of spoken utterances with corresponding call- 

labels is used, followed by using a separate test set in the range of 1000 
utterances. We denote these HHS-train and HHS-test respectively. HHS 
denotes human/human speech-only. 

The ASR phone recognizer 210 that is applied to the training speech is 
15 task-independent. In particular, a phonotactic language model was trained on 

the switchboard corpus using a Variable-Length N-gram Stochastic Automaton. 

This corpus is unrelated to the HMIHY task, except in that they both comprise 

fluent English speech. Off-the-shelf telephony acoustic models may be used. 

Applying the ASR phone recognizer 210 to the HMIHY test speech data yields a 
20 phone accuracy of 43%. The training and test sets so generated are denoted by 

ASR-phone-train and ASR-phone-test respectively. 

For a baseline comparison, a 'noiseless' phonetic transcription was 
generated from the orthographic transcriptions, by replacing each word by its 
most likely dictionary pronunciation and deleting word-delimiters. E.g. "collect 
25 cair is converted to 7C axlehKTKao I" (see Figs. 1 0 and 1 1 , for example). We 
denote these data sets as transcr-phone-train and transcr-phone-test 

The number of recognized phones per utterance is distributed as shown in 
Fig. 6. The mean length is 54 phones per utterance. The shapes of those 
distributions are similar, with a mean duration of 5.9 seconds. 
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For each utterance, the length of the recognized phone sequence is 
compared with the length of the phonetic transcription. These values are scatter- 
plotted in Fig. 7, with the diagonal shown for reference. Observe that in most 
cases, the transcribed and recognized utterances have approximately the same 
5 length. Deviation from the diagonal is in two directions. The above-diagonal 
points correspond to speech being recognized as background noise or silence. 
The below-diagonal points correspond to background noise being recognized as 
speech. 

In step 3200, the salient phone-phrase generator 220 selects candidate 
1 0 phone-phrases from the raw training speech corpus. While the system and 
method of the invention is illustrated and described using the term phone- 
phrases, it is again important to note that phone-phrases are actually sub- 
morphemes that may be acoustic or non-acoustic (i.e., made up of non-verbal 
sub-morphemes such as tablet strokes, gestures, body movements, etc.). 
15 However, as discussed above, for ease of discussion, the systems and methods 
illustrated in the drawings and discussed in the below concern only phone- 
phrases. Consequently, the invention should not be limited to using just phone- 
phrases and should encompass the utilization of any sub-units of any known or 
future method of communication for the purposes of recognition and 
20 understanding. 

Fig. 4 illustrates a more detailed flowchart of the candidate phone-phrase 
selection process that takes place in step 3200. In step 3210, the raw training 
speech corpus is filtered using grammatical inference algorithms, such as those 
defined in U.S. Patent No. 5,675,707. As a result of the filtering process, in step 

25 3220, all observed phone sequences of the predetermined length are selected. 
In step 3230, he salient phone-phrase generator 220 determines whether the 
desired maximum phrase length has been met. If the desired maximum phrase 
length has been met, in step 3240, the salient phone-phrase generator 220 
selects the phone sequence as a candidate phone-phrase. Conversely, if the 

30 desired maximum phrase length has not been met, the process returns to step 
3210 to resume filtering the training speech corpus. 
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Once the candidate phone-phrases have been selected, in step 3300, the 
salient phone-phrase generator 220 selects a subset of the candidate phone- 
phrases. Thus, new units are acquired by the above process of searching the 
space of observed phone-sequences and selecting a subset according to their 
5 utility for recognition and understanding. The resultant subset selected is 
denoted as salient phone-phrases. Examples of salient phone-phrases for the 
word "collect" are shown in Fig. 10. 

The salient phone-phrase generator 220 may perform the selection of 
salient phone-phrases by first using a simplified measure of the candidate phone- 
1 0 phrase's salience for the task as the maximum of the a posteriori distribution, 

^a X (/) = maxPr(C|/), 

where C varies over the 15 call-types in the HMIHY task. The salient 
phone-phrases are then selected by applying a threshold on P max and by using a 
multinomial statistical significance test. This significance test excludes low- 
15 frequency phrases for which a fortunate conjunction of events can give a high 
appearance salience purely by chance. It tests the hypothesis that the observed 
call-type count distribution is a sample from the prior distribution. 

In step 3400, the salient phone-phrases are clustered into acoustic 
morphemes by the clustering device 230. Fig. 1 1 shows examples of acoustic 

20 morphemes for the word "collect". The clustering is achieved using a 

combination of string and semantic distortion measures using methods, for 
example, such as those in U.S. Patent Application No. 09/217,635. Each cluster 
is then compactly represented as a finite state machine and stored in the 
acoustic morpheme database 140. The acoustic morphemes stored in the 

25 acoustic morpheme database 140 may then be used in the task classification 
process shown in Fig. 5. 

Fig. 5 is a flowchart of a possible task classification process using acoustic 
morphemes. The process begins at step 5000 and proceeds to step 5100 where 
input speech recognizer 150 receives an input communication, such as speech, 
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from a user, customer, etc. The input speech may, of course, be expressed in 
verbal speech, non-verbal speech, multimodal forms, or using a mix of verbal 
and non-verbal speech. 

Non-verbal speech may include but are not limited to gestures, body 
5 movements, head movements, non-responses, text, keyboard entries, keypad 
entries, mouse clicks, DTMF codes, pointers, stylus, cable set-top box entries, 
graphical user interface entries and touchscreen entries, or a combination 
thereof. Multimodal information is received using multiple channels (i.e., aural, 
visual, etc.). The user's input communication may also be derived from the 
10 verbal and non-verbal speech and the user's or the machine's environment. 
Basically, any manner of communication falls within the intended scope of the 
invention. However, for ease of discussion, we will focus on verbal speech in the 
examples and embodiments set forth below. 

In step 5200, the acoustic morpheme detector 160 detects acoustic 
15 morphemes that have been recognized by the input speech recognizer 150 using 
the acoustic morphemes stored in the acoustic morpheme database 140. In step 
5300, the task classification processor 170 performs task classifications based 
on the detected acoustic morphemes. The task classification processor 170 may 
apply a confidence function based on the probabilistic relation between the 
20 recognized acoustic morphemes and selected task objectives, for example. In 
step 5400, the task classification processor 170 determines whether a task can 
be classified based on the detected acoustic morpheme. If the task can be 
classified, in step 5700, the task classification processor 170 routes the 
user/customer according to the classified task objective. The process then goes 
25 to step 5900 and ends. 

If the task cannot be classified in step 5400 (i.e. a low confidence level 
has been generated), in step 5500, a dialog module (located internally or 
externally) the task classification process or 170 conducts dialog with the 
user/customer to obtain clarification of the task objective. After dialog has been 
30 conducted with the user/customer, in step 5600, the task classification processor 
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170 determines whether the task can now be classified based on the additional 
dialog. If the task can be classified, the process proceeds to step 5700 and the 
user/customer is routed in accordance with the classified task objective and the 
process ends at step 5900. However, if task can still not be classified, in step 
5 5800, the user/customer is routed to a human for assistance and then the 
process goes to step 5900 and ends 

The example below illustrates this acoustic morpheme generation 
process. Consider a candidate phone-phrase, 

f = P1P2 - .-Pn , 

1 0 where p, are phones. Denote its frequency by F(f) . A measure of its utility for 
recognition is the mutual information of its components, denoted Ml(f), which 
may be approximated via 

MI(f) = MI( Pl p 2 ...p^;p n ) . 

As discussed above, a simplified measure of its salience for the task is the 
1 5 maximum of the a posteriori distribution, 

P mx (/) = maxPr(C|/) , 

where C varies over the 1 5 call-types in the HMIHY task. 

These features for phone-phrases observed in the noise-free case are 
characterized transcr-phone-train. In Fig. 8, the Ml distributions of these phone- 

20 phrases are shown for lengths 2-4. It can be observed that the Ml distributions 
shift positively as length increases, corroborating the increased predictive power 
of longer units. It can also be observed, however, that while many of these 
phrases have positive predictive power (Ml>0), many do not. Thus, for 
constructing larger units from smaller ones, attention is restricted to the positive 

25 tail of these Ml distributions. 

For each of these phone-phrases, P max (f) is computed, which measures 
the salience of a phrase for the task. Fig. 9 shows the distribution of P max for 
varying length phrases. It can be observed that for single phones, P max is near- 
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random, corroborating the intuition that semantics is carried by longer phone- 
phrases. It can also be observed that the positive shift in the distributions as 
length increases. In particular, focus on the region P max > 0.9 , which 
corresponds to highly salient phone-phrases. As length increases, more of these 
5 highly salient phrases are discovered. 

The goal of this process is to grow the phone-phrases until they have the 
salience of words and word-phrases. Thus, the search criteria for selecting 
longer units is a combination of their utility for within-language prediction, as 
measured by Ml, and their utility for the task, as measured by P max . For phrases 
1 0 with large P max ., the Ml of the phrase tends to be larger than average. This 
correlation was exploited successfully for frequency-compensated salience in 
earlier experiments discussed above, but is not exploited here. In the earlier 
experiments, a set of salient phone-phrases of length < 16 was generated via a 
two-pass process as follows: 

1 5 • Select phone-phrases with F(f) >5 and length < 4; 

• Filter the training corpus ASR-phone-train with those phrases, using a left- 
right top-down filter with the phrases prioritized by length. 

• Select subsequences from the filtered corpus of fragment-length <4, (i.e. with 
#phones < 16) and with Ml>1 and P max >0.5. 

20 This particular iterative selection process was selected based on ease of 

implementation should not be considered optimal. The resultant set of salient 
phone-phrases have lengths <16, distributed as shown in Fig. 12. 

An experiment evaluating the utility of these methods in the HMIHY task 
was conducted. A classifier was trained from the output of a phone recognizer 
25 on 7462 utterances, which was denoted ASR-phone-train. Salient phone- 
phrases of length <16 were selected, as described above. The salient phone- 
phrases were then clustered into salient grammar fragments. A single-layer 
neural net was trained with these fragments as input features. The resultant 
classifier was applied to the 1000 utterance test-set, ASR-phone-test. 
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The call-classification results are scored following the methodology of U.S. 
Patent No. 5,675,707. In this method, an utterance is classified by the system as 
one of 15 call-types or rejected as 'other 9 . Rejection is based on a salience- 
threshold for the resulting classification. One dimension of performance is the 
5 False Rejection Rate (FRR), which is the probability that an utterance is rejected 
in the case that the user wanted one of the call-types. The cost of such an error 
is a lost opportunity for automation. The second dimension of performance is the 
Probability of Correct Classification (P c ) when the machine attempts a decision. 
The cost of such an error is that of recovery via dialog. Varying the rejection 
10 threshold traces a performance curve with axes P c and FRR. 

Fig. 13 shows the results of two experiments evaluated on the 1K test set 
ASR~phone-test The first experiment uses acoustic-morphemes trained from 
speech via ASR-phone-train. There is a remarkably high Probability Correct for 
those utterances which are classified ( > 90% ), albeit with a relatively high False 

15 Rejection Rate. The second experiment was trained on text, transcr-phone-train 
and evaluated on speech ASR-phone-test . Interestingly, it is significantly worse 
than training on speech. This corroborates the intuition that when classifying 
'noisy 1 data, it is better to train in matched conditions. Call-classification 
operating curves for this task which were trained on word-transcriptions, 

20 exploiting sophisticated language models for ASR and SLU with utterance 
verification are also shown for comparison purposes. 

As shown in Figs. 1 and 2, the method of this invention may be 
implemented using a programmed processor. However, method can also be 
implemented on a general-purpose or a special purpose computer, a 

25 programmed microprocessor or microcontroller, peripheral integrated circuit 
elements, an application-specific integrated circuit (ASIC) or other integrated 
circuits, hardware/electronic logic circuits, such as a discrete element circuit, a 
programmable logic device, such as a PLD, PLA, FPGA, or PAL, or the like. In 
general, any device on which the finite state machine capable of implementing 

30 the flowcharts shown in Figs. 3-5 can be used to implement the speech 
recognition and understanding system functions of this invention. 
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While the invention has been described with reference to the above 
embodiments, it is to be understood that these embodiments are purely 
exemplary in nature. Thus, the invention is not restricted to the particular forms 
shown in the foregoing embodiments. Various modifications and alterations can 
be made thereto without departing from the spirit and scope of the invention. 
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What is claimed is : 

1 1 . A method of task classification using morphemes which operates 

2 on the task objective of a user, the morphemes being generated by clustering 

3 selected ones of salient sub-morphemes from training speech which are 

4 semantically and syntactically similar, comprising: 

5 detecting morphemes present in the user's input communication; 

6 and 

7 making task-type classification decisions based on the detected 

8 morphemes in the user's input communication. 

1 2. The automated task classification method of claim 1 , wherein the 

2 morphemes include at least one of verbal speech and non-verbal speech. 

1 3. The automated task classification method of claim 2, wherein the 

2 non-verbal speech includes the use of at least one of gestures, body movements, 

3 head movements, non-responses, text, keyboard entries, keypad entries, mouse 

4 clicks, DTMF codes, pointers, stylus, cable set-top box entries, graphical user 

5 interface entries and touchscreen entries. 

1 4. The automated task classification method of claim 1 , wherein the 

2 morphemes are expressed in multimodal form. 

1 5. The automated task classification method of claim 1 , wherein the 

2 user's input communication is derived from the verbal and non-verbal speech 

3 and the user's environment. 

1 6. The automated task classification method of claim 1 , wherein the 

2 morphemes in the user's input communication are derived from the user's 

3 actions, including the user's focus of attention. 

1 7. The automated task classification method of claim 1 , further 

2 comprising entering into a dialog with the user to obtain a feedback response 

3 from the user. 

1 8. The automated task classification method of claim 7, wherein the 

2 user is prompted to provide a feedback response includes additional information 

3 with respect to the user's initial input communication. 
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1 9. The automated task classification method of claim 7, wherein the 

2 user is prompted to provide a feedback response that includes confirmation with 

3 respect to at least one of the set of task objectives determined in the 

4 classification decision. 

1 1 0. The automated task classification method of claim 1 , wherein the 

2 input communication is routed based on the classification decision. 

1 11. The automated task classification method of claim 10, wherein the 

2 task objective is performed after the input communication is routed. 

1 1 2. The automated task classification method of claim 1 , wherein the 

2 method operates in conjunction with one or more communication networks, the 

3 communication networks including a telephone network, the Internet, an intranet, 

4 Cable TV network, a local area network (LAN), and a wireless communication 

5 network. 

1 13. The automated task classification method of claim 1 , wherein the 

2 method is used for customer care purposes. 

1 14. The automated task classification method of claim 1 , wherein the 

2 classification decisions and corresponding user input communications are 

3 collected for automated learning purposes. 

1 1 5. The automated task classification method of claim 1 , wherein the 

2 relationship between the generated morphemes and the predetermined set of 

3 task objectives includes a measure of usefulness of a one of the morphemes to a 

4 specified one of the predetermined task objectives. 

1 1 6. The automated task classification method of claim 1 5, wherein the 

2 usefulness measure is a salience measure. 

1 1 7. The automated task classification method of claim 1 6, wherein the 

2 salience measure is represented as a conditional probability of the task objective 

3 being requested given an appearance of the morpheme in the input 

4 communication, the conditional probability being a highest value in a distribution 

5 of the conditional probabilities over the set of predetermined task objectives. 
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1 18. The automated task classification method of claim 1 6, wherein 

2 each of the plurality of generated morphemes has a salience measure exceeding 

3 a predetermined threshold. 

1 1 9. The automated task classification method of claim 1 , wherein the 

2 relationship between the generated morphemes and the predetermined set of 

3 task objectives includes a measure of commonality within a language of the 

4 morphemes. 

1 20. The automated task classification method of claim 19, wherein the 

2 commonality measure is a mutual information measure. 

1 21 . The automated task classification method of claim 20, wherein 

2 each of the plurality of generated morphemes has a mutual information measure 

3 exceeding a predetermined threshold. 

1 22. The automated task classification method of claim 1 1 , wherein the 

2 step of making a classification decision includes a confidence function. 

1 23. The automated task classification method of claim 1 1 , wherein the 

2 input communication from the user represents a request for at least one of the 

3 set of predetermined task objectives. 

1 24. The automated task classification method of claim 1 1 , wherein the 

2 input communication is responsive to a query of a form "How may I help you?". 

1 25. The automated task classification method of claim 1 1 , wherein 

2 each of the verbal and non-verbal speech are directed to one of the set of 

3 predetermined task objectives and each of the verbal and non-verbal speech is 

4 labeled with the one task objective to which it is directed. 

1 26. A method of task classification which operates on the task objective 

2 of a user, comprising: 

3 selecting salient phone-phrases from training speech; 

4 generating acoustic morphemes by clustering selected ones of the 

5 salient phone-phrases which are semantically and syntactically similar; 

6 detecting acoustic morphemes present in the user's input 

7 communication; and 
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8 making task-type classification decisions based on the detected 

9 acoustic morphemes in the user's input communication. 

1 27. A method of task classification which operates on the task objective 

2 of a user, comprising: 

3 selecting salient sub-morphemes from training speech; 

4 generating morphemes by clustering selected ones of the salient 

5 sub-morphemes which are semantically and syntactically similar; 

6 detecting morphemes present in the user's input communication; 

7 and 

8 making task-type classification decisions based on the detected 

9 morphemes in the user's input communication. 

1 28. A method of task classification using acoustic morphemes which 

2 operates on the task objective of a user, the acoustic morphemes being 

3 generated by clustering selected ones of salient phone-phrases from training 

4 speech which are semantically and syntactically similar, comprising: 

5 detecting acoustic morphemes present in the user's input 

6 communication; and 

7 making task-type classification decisions based on the detected 

8 acoustic morphemes in the user's input communication. 
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Abstract of the Disclosure 

The invention concerns a method of task classification using morphemes 
which operates on the task objective of a user. The morphemes may be 
5 generated by clustering selected ones of the salient sub-morphemes selected 
from training speech which are semantically and syntactically similar. The 
method may include detecting morphemes present in the user's input 
communication, and making task-type classification decisions based on the 
detected morphemes in the user's input communication. The morphemes may 
10 be verbal and/or non-verbal. 
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IN THE UNITED STATES 
PATENT AND TRADEMARK OFFICE 

Declaration and Power of Attorney 

As a below named inventor, I hereby declare that: 

My residence, post office address and citizenship are as stated below next to my name. 

I believe I am an original, first and joint inventor of the subject matter which is claimed and 
for which a patent is sought on the invention entitled METHOD FOR TASK CLASSIFICATION 
USING MORPHEMES the specification of which is attached hereto. 

I hereby state that I have reviewed and understand the contents of the above identified 
specification, including the claims, as amended by an amendment, if any, specifically referred to in 
this oath or declaration. 

I acknowledge the duty to disclose all information known to me which is material to 
patentability as defined in Title 37, Code of Federal Regulations, 1.56. 

I hereby claim foreign priority benefits under Title 35, United States Code, 1 19 of any foreign 
application(s) for patent or inventors^ certificate listed below and have also identified below any 
foreign application for patent or inventors^ certificate having a filing date before that of the 
application on which priority is claimed: 

None 

I hereby claim the benefit under Title 35, United States Code, 120 of any United States 
application(s) listed below and, insofar as the subject matter of each of the claims of this application 
is not disclosed in the prior United States application in the manner provided by the first paragraph 
of Title 35, United States Code, 1 12, we acknowledge the duty to disclose all information known to 
us to be material to patentability as defined in Title 37, Code of Federal Regulations, 1.56 which 
became available between the filing date of the prior application and the national or PCT 
international filing date of this application: 



Serial No. 60/163,838 
Filed November 5, 1999 
Status: PENDING 



2 



I hereby declare that all statements made herein of my own knowledge are true and that all 
statements made on information and belief are believed to be true; and further that these statements 
were made with the knowledge that willful false statements and the like so made are punishable by 
fine or imprisonment, or both, under Section 1001 of Title 18 of the United States Code and that 
such willful false statements may jeopardize the validity of the application or any patent issued 
thereon. 



I hereby appoint the following attorney(s) with full power of substitution and revocation, to 
prosecute said application, to make alterations and amendments therein, to receive the patent, and to 
transact all business in the Patent and Trademark Office connected therewith: 



Samuel H. Dworetsky(Reg. No. 27873) 
Thomas A. Restaino (Reg. No. 33444) 
Cedric G. DeLaCruz (Reg. No. 36498) 
Michele L. Conover (Reg. No. 34962) 
Rohini K. Garg (Reg. No. 45272) 
Benjamin S. Lee (Reg. No. 42787) 
Robert B. Levy (Reg. No. 28234) 
Susan E. McHale (Reg. No. 35948) 
Alfred G. Steinmetz (Reg. No. 22971) 



I also appoint the registered practitioners of Kenyon & Kenyon included in the Customer 
Number provided below as associate attorneys, to prosecute this application and transact all business 
in the Patent and Trademark Office connected therewith. 



CUSTOMER NUMBER: 23838 



Please address all correspondence to Ronald E. Prass, Jr., Kenyon & Kenyon, Suite 700, 1500 
K Street NW, Washington, D.C. 20005. Telephone calls should be made to Mr. Prass by dialing 
202-220-4200. 



Full name of I s joint inventor: Allen Louis Gorin 



Inventor's signature 




Residence: Berkley Heights, Union County, New Jersey 

Citizenship: United States of America 

Post Office Address: 205 Spring Ridge Drive 

Berkely Heights, New Jersey 07922 
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Full name of 2 nd j oint inventor: Dij ana Petro vska-Delacretaz 

Inventor's signature Date 

Residence: Paris, France 
Citizenship: Switzerland 

Post Office Address: 46, Rue de Barrault 

75634 Paris, Cedex 13 France 

Full name of 3 rd joint inventor: Giuseppe Riccardi 



Inventor's signature ^(M^^ Date t 6 //6^* 



Residence: Hoboken, Hudson County, New Jersey 
Citizenship: Italy 

Post Office Address: 716 Hudson Street, Apt. 1C 

Hoboken, New Jersey 07030 

Full name of 4 th joint inventor: Jeremy Huntley Wright 

Inventor's signature r ^^ rt ^^ LD^^^ Date t&^t/ej 

Residence: Warren, Somerset County, New Jersey 
Citizenship: Great Britain 

Post Office Address: 87 King George Road 

Warren, New Jersey 07059 
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IN THE UNITED STATES 
PATENT AND TRADEMARK OFFICE 

Declaration and Power of Attorney 

A$ a below named inventor, I hereby declare that: 

My residence, post office address and citizenship are as stated below next to my name. 

I believe I am an original, first and joint inventor of the subject matter which is claimed and 
for which a patent is sought on the invention entitled METHOD FOR TASK CLASSIFICATION 
USING MORPHEMES die specification of which is attached hereto. 

I hereby state that I have reviewed and understand the contents of the above identified 
specification, including the claims, as amended by an amendment; if any, specifically referred to in this 
oath or declaration. 

I acknowledge die duty to disclose all information known to me which is material to 
patentability as defined in Tide 37> Code of Federal Regulations, L56. 

I hereby claim foreign priority benefits under Title 35, United States Code, 119 of any foreign 
applications) for patent or inventors= certificate listed below and have also identified below any 
foreign application for patent or inventors- certificate having a filing date before that of the 
application on which priority is claimed; 

None 

1 hereby claim the benefit under Title 35, United States Code, 120 of any United States 
application^) listed below and, insofar as die subject matter of each of die claims of dm application is 
not disclosed in die prior United States application in die manner provided by the first paragraph of 
Title 35, United States Code, 1 12, we acknowledge the duty to disclose all information known to us to 
be material to patentability as defined in Title 37, Code of Federal Regulations, 1.56 which became 
available between the filing date of the prior application and the national or PCT international filing 
date of this application: 

Serial No, 60/163,838 
Filed November 5, 1999 
Status: PENDING 
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I hereby declare that all statements made herein of my own knowledge are true and that all 
statements made on information and belief are believed to be true; find furdier rhat these statements 
were made with die knowledge that willful false statements and the like so made are punishable by fine 
or imprisonment, or both, under Section 1001 of Title 18 of the United States Code and diat such 
willful false statements may jeopardise the validity of the application or any patent issued diereon. 

I hereby appoint the following attorneys) with full power of substitution and revocation, to 
prosecute said application, to make alterations and amendments therein, to receive the patent, and to 
transact all business in the Patent and Trademark Office connected therewith: 



1 also appoint the registered practitioners of Kenyon <Sl Kenyon included in the Customer 
Number provided below as associate attorneys, to prosecute this application and transact all business 
in the Parent and Trademark Office connected therewith. 



Please address all correspondence to Ronald E- Prass, Jr., Kenyon &l Kenyon, Suite 700, 1500 
K Street NW, Washington, D.C. 20005. Telephone calls should be made to Mr. Prass by dialing 202- 
2204200. 



Samuel M. Dworetsky(Reg. No. 27873) 
Thomas A. Restaino (Reg. No. 33444) 
Cedric G- D*LaCruz (Reg. No. 36498) 
Michele L Conover (Reg. No, 34962) 
Rohini K. Garg (Reg, No, 4$272) 
Benjamin S. Lee (Reg. No. 42787) 
Robert B. Levy (Reg, No. 28234) 

Susan E. McHale (Reg. No. 35948) 
Alfred O. Steimncte (Reg. No. 2297 1) 
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Full name of l Ar joinr inventor: Allen Louis Gorin 



Inventor's signature 



Date 



Residence: Berkley Heights, Unioa County, New Jersey 



Citizenship: United States of America 



Post Office Address: 



205 Spring Ridge Drive 

Berkely Heights,, New Jersey 07922 
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Full name of 2 nd joint inventor; Dijana Petxovska-Deiacretaz 
Inventor's signature 

Date &Ockfii*&*>< 



Residence: Paris, France 
Citizenship: Switzerland 
Post Office Address: 



Vu pour certification ' 
simplement materielle 

46, Rue de Barrauit <j e signature 

75634 Paris, Cedex 13 France 



Full name of" 3* 1 joint inventor: Giuseppe Riccardi 
Inventor's signature 



Residence: Hoboken, Hudson County, New Jersey 
Citizenship; Italy 



Post Office Address; 



7 1 6 Hudson Street, Apt, 1C 
Hoboken, New Jersey 07030 



Full name of 4* joint inventor: Jeremy Hu ntiey Wright 
Inventor's signature 



Residence: Warren, Somerset County, New Jersey 
Citizenship: Great Britain 



Post Office Address; 



87 King George Road 
Warren, New Jersey 07059 




Date 



Date 
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